On Robustness/Performance Tradeoffs in Linear Programming and Markov Decision Processes

Computation of a satisfactory policy for a decision problem when the parameters of the model are uncertain is a problem encountered in many applications. The traditional robust approach is based on a worst-case analysis and may lead to overly conservative solutions. In this paper we directly quantify the robustness to uncertainty and consider the tradeoff between the nominal performance and robustness measures. Optimization in both linear programming and Markov decision processes is discussed. For linear programming we consider the tradeoff between the nominal cost of a solution and a robustness measure that quantifies the magnitude of constraint violation under the most adversarial parameters. We propose an algorithm that computes the whole set of Pareto efficient solutions based on parametric linear programming. For Markov decision processes, we consider the tradeoff between the performance under nominal parameters and the performance under adversarial parameters. For the special case where only the rewards are uncertain, we propose an algorithm that computes the whole set of Pareto efficient policies in a single pass.